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foreword 


The PISA data base is open to the world - a powerful 
resource for everyone who wants to understand 
education and help improve it, from policy-makers 
and researchers to school leaders, teachers and 
parents. Over recent years, McKinsey have done just 
this, drawing on PISA to identify the policies and 
practices that make a real difference. Their work 
began with ground-breaking reports on The World's 
Best School Systems And How To Build Them. 

And these new regional analyses of student-level 
performance represent another significant milestone. 
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The reports suggest that students' attitudes and 
motivation are critical drivers of achievement. So too are their 
experience in the classroom, of both teaching strategies and 
digital technology, as well asthe time they spend in education. 
McKinsey's perceptive insights will encourage schools 
around the world to discover new ways to nurture and inspire 


their students. 


What sets these reports apart is their regional focus. I often 

hear countries say that learning from the world's outstanding 
systems is vital, but thatjust as powerful is the chance to learn 
from their own neighbours, with similar cultural backgrounds 


and with shared problems and opportunities. 


Inevery country, the search is on for ways to take education 
tothe next level, to prepare young people for a dramatic and 
challenging century. This is complex work. Whatis the right 


mix of policies, implementation strategies and enabling 
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conditions - in each country and region? How should they 

be prioritised, sequenced and linked? If we are really to 
secure achievement, well being and equity, on a global basis, 
then these will be the issues that educators need to work on. 
The new reports from McKinsey offer usa fresh and welcome 
perspective. 


Andreas Schleicher 
Director for the Directorate of Education and Skills | OECD 


executive 
summary 


A well-educated citizenry is an economic and social 
necessity. But there is little consensus about what it 
takes to deliver a quality education. 


In two previous reports, one on the world’s best- 
performing school systems (2007) and the other on the 

most improved ones (2010), we examined what great school 
systems look like, and how they can sustain significant 
improvements from any starting point. In this report, we 
switch our focus from systems to student-level performance, 
by applying advanced analytics and machine learning to the 
results of the Organisation for Economic Co-operation and 
Development’s (OECD) Program for International Student 
Assessment (PISA). Beginning in 2000 and every three 
years since, the OECD has tested 15-year-olds around the 
world on math, reading, and science. It also surveys students, 
principals, teachers, and parents on their social, economic, 


andattitudinal attributes. 


Usingthis rich data set, we have created five regional reports 
that consider what drives student performance. In Latin 
America, ten countries participated in the 2015 PISA: 
Argentina, Brazil, Chile, Colombia, Costa Rica, Dominican 
Republic, Mexico, Peru, Trinidad and Tobago, and Uruguay. 
Onthe whole, Latin America's PISA scores have improved, 
but the region still lags not only the OECD average but also 


other countries ata similar economic level. 


This research is not intended as a roadmap to system 
improvement; that was the theme of our 2010 report, which 
setoutthe interventions school systems need to undertake 
to move from poor to fair to good to great to excellent 
performance. Instead, this report examines five specific 
factors that we found to be particularly important to student 
outcomes: mindsets, teaching practices, information 
technology, hours of instruction, and early-childhood 


education. 
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The report's findings include the following 
five highlights: 


Student mindsets have almost double the 
effect of socioeconomic background on 
outcomes: 


Itis hardly news that students' attitudes and beliefs—what 
we term their “mindsets” — influence their academic 
performance. The magnitude ofthis effect, and which 
mindsets matter most, is still under debate; and it is here 

that we focused our research. While there is likely a linkage 
between socioeconomics and student mindsets, we measured 
the effect of mindsets that is not explained by socioeconomics 
alone. By analyzingthe PISA data, we found that mindset 
factors have almost double the predictive power (30 percent) 
compared to home environment and demographics (16 
percent) on student PISA scores in Latin America. This 
relationship also holds true in all other regions, which 


reinforces the importance of this finding. 


Some mindsets are more important than others. For example, 
we compared motivation calibration (being able to identify 
what motivation looks like in day-to-day life, including 
"working on tasks until everything is perfect” and “doing 
more than what is expected") to self-identified motivation 
(“wanting to be the best,” and “wanting to get top grades"). 

In the 2015 PISA assessment, motivation calibration has 
more than twice the impact of self-identified ambition. 
Students who had good motivation calibration scored 14 
percent (or 55 PISA points) higher on the science test than 
poorly calibrated ones. The relationship is particularly strong 
for students in poorly performing schools, where having a 
well-calibrated motivation mindset is equivalent to vaulting 
into a higher socioeconomic status. In these schools, students 


inthe lowest socioeconomic quartile who are well-calibrated 
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^ 


perform better than those in the highest socioeconomic 


quartile who are poorly calibrated. In contrast, students with 
high self-identified motivation score just six percent higher 


than those without. 


Other general mindsets that are predictive of student 
outcomes include havinga strong sense of belonging at school 
and havinglow test anxiety. We also found that students with 
astrong growth mindset (those who believe they can succeed 
ifthey work hard) outperform students with a fixed mindset 
(those who believe that their capabilities are static) by 12 
percent. Havinga growth mindset was particularly predictive 
for students in poorly performing schools, for those in lower- 


income quartiles, and for boys. 


The prevalence of beneficial mindsets varies between boys 
and girls. While girls are more likely to have strong motivation 
calibration and instrumental motivation, they are also more 


likely to have high levels oftest anxiety. 


To be clear, mindsets alone cannot overcome economic and 
social barriers, and researchers still debate the extent to which 
school-system-level interventions can shift student mindsets. 
Our research does, however, suggest that they matter a great 
deal, particularly for those livingin the most challenging 
circumstances. The research on this subject is both nascent 
and predominantly U.S.-based. Considering its importance, 
local experimentation in Latin America and elsewhere should 


bea priority. 


Students who receive a blend of 
inquiry-based and teacher-directed 
instruction have the best outcomes: 


High-performing and fast-improving school systems require 
high-quality instruction. It's that simple. And that difficult. 
We evaluated two types of science instruction to understand 
how different teaching styles affect student outcomes. 

The first type is *teacher-directed instruction” where the 
teacher explains and demonstrates scientific ideas, discusses 
questions, and leads classroom discussions. T he second 

is “inquiry-based teaching," where students play a more 
active role, creating their own questions and engaging in 
experiments. Our research found that student outcomes are 
highest with a combination ofteacher-directed instruction 
in most or almost all classes, with inquiry-based teaching in 
some classes. If all students experienced this blend, average 
PISA scores in Latin America would rise 19 PISA points, 
equivalent to over half a school year of learning. 


Given the strong support for inquiry-based pedagogy, this 
scems counterintuitive. We offer two hypotheses for these 
results. First, students cannot progress to inquiry-based 
methods without a strong foundation of knowledge, gained 
through teacher-directed instruction. Second, inquiry- 
based teaching is inherently more challenging to deliver, 
and teachers who attempt it without sufficient training and 
support will struggle. Better teacher training, high-quality 
lesson plans, and school-based instructional leadership can 


help. It’s also important to note that some kinds of inquiry- 
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In poorly performing 
schools having a 
well-calibrated 
motivation mindset 

is equivalent 

to vaulting from the 
lowest to the highest 
socioeconomic quartile. 


based teaching are better than others. For example, explaining 
how a science concept can be applied to a real-world situation 
appears to boost outcomes, whereas having students design 


their own experiments does the opposite. 


While technology can support student 
learning outside of school, its record inside 
school is mixed. The best results come 
when technology is placed in the hands 

of teachers: 


Screens are not the problem when it comes to student 
outcomes- but neither are they the answer. Our research 
examined the impact of first exposure to information and 
communications technologies (ICT), and the impact of IC'T 
for 15 year olds - at home and also during school. Students 
with their first digital exposure before the age of 6 score 

45 PISA points higher than those exposed at age 13 or later 
(controlling for socioeconomic status, school type, and 
location). Higher socioeconomic status students are more 
likely to start using devices at an early age, which has worrying 


implications for the equity gap. 


At home, two to four hours of Internet use per day for 15-year- 
olds is associated with the highest science performance, 46 
PISA points higher than for students with no after-school 
Internet use (again, after controlling for socioeconomic 
status, school type, and location). Notably, more than half 

the benefit of after-school use is captured with just 31 to 60 
minutes of Internet use a day. There appears to be declining 
impact (and possibly negative behavioral implications) when 


students spend four hours or more a day before a screen. 


The impact of ICT use on students during the school day is 
much more mixed: from minus-40 to plus-46 PISA points, 
depending on the type of hardware. Most important, we 
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found that deploying ICT to teachers, rather than students, 
works best. For example, adding one data projector per 
classroom leads to a marked increase on students' PISA 
science performance, over 30 times as much as adding one 
student computer to the same classroom. Some student-based 
classroom technologies, such as tablets and e-book readers, 
actually appear to hurt Latin American student performance. 
These results describe the impact of education technology 

as currently implemented, not its eventual potential. They 
evaluate only hardware, not software, and do not account 
forrapid evolution. Even so, Latin American leaders should 
not assume the impact of ICT will always be positive or 

even neutral. Systems should ensure that ICT programs are 
fully integrated with curriculum and instruction, and are 


supported by teacher professional development and coaching. 


Increasing the school day improves 
outcomes - up to seven hours per day. 
Significant gains can also be made from 
using existing time better: 


School facilities are stressed in many parts of Latin America, 
with buildings often hosting two shifts of students daily. One 
consequence is that many students simply don't spend that 
much time in school. Although the regional average is five 
hours per day, 15 percent of students are in school 4.5 hours 

a day or less. A number of countries, including Brazil and 
Colombia, are therefore seeking to end school sharing and 


extend the school day. 


It makes intuitive sense that spending more time in school 
should improve performance, and the PISA results bear 

that assumption out. Across Latin America, PISA science 
outcomes increase by 3.7 percent (or 14 PISA points) for every 
30 minutes of additional daily classroom instruction, up to 


seven hours per day. If all students performed at the level of 


those currently receiving 6.5 to 7 hours ofinstruction per 

day, this would boost average science achievement by about 
35 PISA points across the region. But extending the school 
day is costly, requiring more infrastructure and teachers. 
Another option to increase hours of instruction is lengthening 
the school year; however, recent research in Mexico showed 
diminishing marginal returns to additional days in school 

at present quality levels. Indeed, Latin American countries 
are among the least productive of all the countries that take 
PISA interms of PISA points per hour-in-school, partly 
because only 65 percent of classroom time is actually used 

for learning in several countries; versus an OECD benchmark 
of 85 percent. Ultimately, systems will need to both increase 
the number of hours in school and the quality of each hour. 
Inthe short term, they can make important strides in 
improving student learning for every existing hour in school 
by ensuring they are minimizing non-instructional time, and 
by raisingteacher quality through coaching and professional 


development. 


Early childhood education had a positive 
academic impact on today's 15 year olds, 
however low income students received less 
benefit than high income ones: 


Many studies have shown that quality early-childhood 
education (ECE) improves social and academic outcomes, 
although there are some concerns about fade-out in later 
years. Our findings, like other research, validate the overall 
positive impact of ECE at age 15, but show that there is a 


trade-off between increasing access and ensuring quality. 


On the whole, students with some ECE perform eight percent 
better on the PISA science test a decade later, but there 
are troubling differences among students from different 


backgrounds. High socioeconomic status children get over 
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double the benefit of lower socioeconomic status children, and 
they benefit at every age, with the highest scores when starting 
at age two. Lower socioeconomic status children have the 
highest scores when starting at age four. This highlights the 
importance of investing in good quality ECE especially for 


lower income children. 


As we share these five findings, we are mindful of their limits. 
One cannot find definitive answers from a single source, no 
matter how broad or well designed. The direction of causality, 
sample sizes, missing variables, and nonlinear relationships 
are other issues. There are still many questions that need 

to beresolved through a thoughtful research agenda and 
longitudinal experimentation. T hat said, we believe that 
these five findings provide important insights into how 
students succeed—and that Latin American educators should 


incorporate them into their school improvement programs to 


deliver the progress that their students deserve 
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introduction 


Effective education is 
essential to forge economic 
productivity, address 
inequality, and prepare 
children for constructive 
citizenship. No wonder, then, 
that there is broad interest in 
understanding how to build 
school systems that serve 
everyone well, regardless 

of background, and how to 
improve systems that are not 
making the grade. 


For the past decade, McKinsey has studied these issues. 
In 2007, we published How the world’s best-performing school 
systems come out on top, which examined why some school 
systems consistently perform better than others. This report 
highlighted the importance of getting the right people to 
become teachers, developing their skills, and ensuring that 
the system is able to offer the best possible instruction to 
every child. In 2010, How the world s most improved school 


systems keep getting better explored what it takes to achieve 
significant and sustained performance improvement. This 
report defined poor, fair, good, great and excellent systems 
(see the analytical appendix for more detail) and outlined 
what school systems need to do to progress from one 
performance level to the next! (Exhibit 1). 


These two reports focused on interventions at the system 
level. Inthis report, weundertake a quantitative analysis at 
the student level. To do so, we applied advanced analytics 
and machine learning to develop insights from the world's 
deepest and broadest education data set, the Program 

for International Student Assessment (PISA), run by the 
Organisation for Economic Co-operation and Development 
(OECD). 


Begun in 2000 and repeated every three years since, PISA 
examines 15-year-olds on applied mathematics, reading, and 
science. The most recent assessment, in 2015, covered nearly 
540,000 students in 72 countries. PISA test-takers also 
answer arich set of attitudinal questions; students, teachers, 
parents, and principals completed surveys that provided 
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EXHIBIT 01: OUR 2010 REPORT OUTLINED WHAT INTERVENTIONS ARE 
REQUIRED AT EACH STAGE OF THE SCHOOL SYSTEM IMPROVEMENT JOURNEY 


IMPROVEMENT 
JOURNEY 


THEME 


INTERVENTION 
CLUSTER 


COMMON ACROSS 
ALL JOURNEYS 


POOR TO FAIR 


Achieving the basics 
of literacy and 
numeracy 


Providing motivation 
and scaffolding for low 
skill teachers 

Scripted teaching materials 
External coaches 
nstructional time on task 
School visits by center 
ncentives for high 
performance 


Getting all schools to 

a minimum quality level 
Outcome targets 
Additional support for low 
performing schools 
School infrastructure 
improvement 

Provision of textbooks 


Getting students 
in seats 

Expand school seats 
Fulfil students’ basic 
needs to raise 
attendance 


FAIR TO GOOD 


Getting the 
foundations 
in place 


Data and accountability ° 


foundation 
Transparency to schools 
and/or public on school 
performance 

School inspections 

and inspections 
institutions 


Financial and 
organizational 
foundation 
Optimization of school 
and teacher volumes 
Decentralizing financial 
and administrative rights 
Increasing funding 
Funding allocation model 
Organizational 

redesign 


Pedagogical 
foundation 
School model/ 
streaming 
Language of 
instruction 


GOOD TO GREAT 


Shaping the 
professional 


Raising calibre of 
entering teachers 
and principals 
Recruiting programs 
Pre-service training 
Certification 
requirements 


Raising calibre of 
existing teachers and 
principals 

In-service training 
programs 

Coaches 

Career tracks 

Teacher and 
community forums 


School-based 
decision-making 
Self-evaluation 
Independent and 
specialized schools 


GREAT TO EXCELLENT 


Improving through 
peers and innovation 


* Cultivating peer-led 
learning for teachers 
and principals 

— Collaborative practice 

— Decentralizing 
pedagogical rights 
to schools & teachers 

— Rotation and 
secondment programs 


* Creating additional 
support mechanisms 
for professionals 

— Release profession- 
als from admin burden 
by providing additional 
administrative staff 


* System-sponsored 
experimentation/ 
innovation across 
schools 

— Providing additional 
funding for innovation 

— Sharing innovation from 
front-line to all schools 


Six interventions: [1] Revising curriculum and standards; [2] Reviewing reward and remunerations 
structure; [3] Building technical skills; [4] Assessing student performance; [5] Utilizing student learning 
data, and [6] Establishing policy documents and education laws 


EXHIBIT 02: 


OECD PISA test performance - survey data 


information on home environment, economic status, student mindsets and behaviors, school resources and 
leadership, teaching practices, teacher background, and professional development (Exhibit 2). The 2015 
PISA focused on scientific performance, with half of the student assessment related to science, and the other 
half split between reading and math?. The survey questions therefore largely addressed science teaching 


and learning. 


Standardized tests have their shortcomings. They cannot measure important soft skills or non-academic 
outcomes, and they are subject to teaching-to-the-test and gaming the system. Even so, we believe that 
PISA provides powerful insights on global student performance, especially because it aimsto test the 


understanding and application of ideas, rather than facts derived from rote memorization. 


In this report, we examine educational performance in Latin America, specifically in terms of the ten 
countries in the region that took PISA in 2015: Argentina, Brazil, Chile, Colombia, Costa Rica, Dominican 
tepublic, Mexico, Peru, Trinidad and Tobago, and Uruguay.* While we concentrate on the 2015 PISA 


results, we also consider previous ones, using a range of traditional and advanced analytical techniques. 


First, we used a supervised machine learning and feature 
discovery tool that identified variables and groups of variables 
that were most predictive of student performance. We then 
applied more traditional descriptive and statistical analyses to 
factors that were shown to be most important in contributing 
to students’ PISA performance. (For more, see the analytical 


appendix at the end of the report.) 


We looked not only at macro performance, but also at how 
patterns differed by the system performance levels outlined 
in our 2010 report, and by students’ economic, social, and 
cultural status (or ESCS; see the analytical appendix for 

an explanation). Our research resulted in five key findings, 
regarding mindsets, duration of classroom instruction, 


teaching practices, information technology, and early- 
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childhood education. These five findings emerged as both 


highly predictive of student performance and potentially 
responsive to school system interventions, and therefore 


should be subject to further exploration. 


In what follows, we first examine Latin America's education 
performance in historical terms and then discuss each of 
the five findings, before suggesting possible implications 
for school systems. Our intention is to offer insights 


that policymakers and practitioners can use to make 


im proveme nts 


al 


setting the 
context: 


Latin America’s 


educational performance 


Educational performance 

can be measured in terms of 
overall quality (absolute 
scores), cost-effectiveness 
(performance per dollar spent) 
and equity (differences in 
performance between boys 
and girls, and among different 
economic and ethnic groups). 
In terms of overall quality, Latin America's PISA scores are 


well below the OECD average in math, reading, and science 
(Exhibit 3). 
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EXHIBIT 03: 


Science 2015 Reading 2015 Math 2015 


EXHIBIT 04: LATIN AMERICAN COUNTRIES PERFORM WORSE 
THAN EXPECTED GIVEN THEIR GDP PER CAPITA 


PISA Science 2015 . E Excelent Mi Great MGood W Far f Poor 
Mean Score Countries above the 


trendline achieved better | 
results than expected, 
given their GDP 
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340 | 
|] | i 
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320 | * Dominican Republic 
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GDP per capita PPP USD 
Source: PISA 2015 


When compared with global peers with similar levels of GDP and education spending, Latin America 
does not perform well either, suggesting low cost-effectiveness (Exhibit 4 and 5). 


Even Latin American countries that perform relatively well are struggling with equity. The percent of 


score explained by economic, social, and cultural status is more than 15 percent in Chile, Costa Rica, and 
Uruguay, and more than 20 percent in Argentina and Peru. The global average is 13 percent (Exhibit 6). 
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EXHIBIT 05: OTHER COUNTRIES AND REGIONS ACHIEVE 
BETTER RESULTS AT SIMILAR LEVELS OF SPENDING 


2015! PISA science E Excellent Mi Great Good W Far E Poor 
Mean Score 


600 


580 
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| Uruguay | Romania. Argentina! | | | | | 
420 | | M Costa Rica "Thailand i rTüriqad & Tobago ied | 
k t Jordan- Colombia | y, | l maa 
400 | Pede Indonesia d Mexico |^ : I H 
| Tunisia | | 
380 ern d LAlgeria I I 
| ] | i 
360 || | | | | | 
340 | > oe | l ! i 
| Dominican | i | 
320 | Republic f | i l 
i ! i | 
300 Es | 
0 1,000- 2,000- 3,000-  4,000- 5,000-  6,000-  7,000- 8000- 49,000 


1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000 
Public spend per student, PPP USD 


1 If 2015 was not available, most recent year was used. 
Source: Global Insight; IMF; PISA; UNESCO; World Bank EdStats; McKinsey analysis 
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EXHIBIT 06: 


performance 


Below average impact of 
soco-economic background 


Above average performance 
Above average impact of 
soco-economic background 


| e Chile 
Trinidad 8 Tobago Argentina + Uruguay 
o ! 
: | € Costa Rica 
Mexico € ** Colombia 


Brazil € 


| @ Peru 


Dominican Republic e 


Below average performance 


Above average impact of 
Soco-economic background 


Below average performance i 
Below average impact of 
Soco-economic background H 


OECD average % of variance 
explained by ESCS (13%) 


Looking beyond PISA, data from UNESCO show that enrollment rates in Latin America drop off sharply 
at the upper secondary level (typically age 15 or 16). This leaves more than 14 million Latin American 


children out of school (Exhibit 7). 


Over the past several PISA cycles, Latin American reading, science, and math scores have improved. 
Reading has improved the most, up 6.6 percent change between 2006 and 2015. Science scores increased 
4.] percent and math 2.5 percent (Exhibit 8). Itbears remembering that these improvements are from a 


low base, and are not enough to pull the continent as a whole out ofthe “poor” performance band. 


EXHIBIT 07: 


WITH SECON 


Net enrollment rate by level of education,! 2015? 


(76) 
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1 As defined according to the International Standard Classification of Education (ISCED): pre-primary: ISCED level O includes preschool and kindergarten programs; primary: 
ISCED level 1 typically begins between ages 5 and 7 and lasts for 4-6 years; lower secondary: ISCED level 2 begins around the age of 11, the equivalent of intermediate 
school, middle school, or junior high school; upper secondary: ISCED level 3 immediately follows lower-secondary education and includes general (academic), technical, 


and vocational education, 


the equivalent of high school. 


If 2015 was unavailable, latest data available were used. 


2 
Source: UNESCO 
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B :: READING AND SCIENCE SCORES HAVE IMPROVED MORE 
THAN MATH 1SCORES SINCE 2006 


Change in average Latin America PISA score over time E Fair E Poor 
PISA score 


450 
440 
430 
420 
410 
400 
390 
380 
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360 


350 


2006 2009 2012 


Source: PISA 2015 


"E A 


Looking at the country level, Colombia and Peru stand out for sustained performance improvements. 
Only Chile, however, has climbed out of the “poor” performance band (Exhibit 9). 


Interms ofequity, there have been some meaningful improvements in narrowing the performance gap 
between wealthy and poor students since 2006 (Exhibit 10).° 


These improvements are heartening, but the pace is slow. In our 2010 report, we showed that fast- 
improving systems moved up to the next level—that is, from poor to fair to good to great to excellent— 
every six years. At current rates of progress, even Latin America’s top improvers will take 10 to 12 years 
to reach the next level. 
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EXHIBIT 09: IMPROVEMENT HAS BEEN UNEVEN 


PISA composite score (average of science, reading, and math scores) E Fair E Poor 


445 
440 
435 
430 
425 
420 
415 
410 
405 
400 
395 
390 
385 
380 
375 
370 


330 


2003 2006 2009 2012 2015 


1 Argentina results in 2015 have sampling issues therefore trend should not be fully trusted. 
Source: PISA 2015 


For school systems in the poor-to-fair stage ofimprovement—meaning everywhere in Latin America 
except Chile—the 2010 report defined three priorities: enrolling students, getting all schools to a 
minimum quality level, and providing motivation and training for low-skilled teachers. In addition, 
we highlighted six interventions that are important at every stage: revising curriculum and standards; 
reviewingreward and remunerations structure; building technical skills; assessing students; using 
student learning data, and establishing policy documents and education laws. 


The insights from this report delve deeper on a few of these system priorities, and add new insights gained 
from student level analysis. What mindsets are most beneficial for students? What does great teaching 
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EXHIBIT 10: ALL LATIN AMERICAN COU 
IN BOTH 2006 AND 2015 NARROW 


RIES THAT TOOK PISA 
D THE EQUITY GAP 


4 


look like? How long should students stay in school? 
What is the role of technology? When should 
education begin? The following five findings, 
based on the PISA data, complement our previous 


work by exploring these questions O 
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have almost double 
the impact of 
socioconomic 
background on 
outcomes 


The role of mindsets in educational achievementis a 
nascent but intriguing field of study. In her 2006 book, Mindset: 
The New Psychology of Success, Carol Dweck argued that 
individuals with “growth mindsets"—thatis, those who believed 
that their success was due to hard work and learning—were more 
resilient and likely to be motivated to succeed than those with 
"fixed mindsets"—those who believed that their innate abilities 
were static and could not be developed. Dweck also argued that 
growth mindsets could be taught. A large-scale 2016 Stanford 
study of all 10th graders in Chile—the largest to date—found 
that having a strong growth mindset rivals socioeconomic status 
in predicting achievement, and that low-income students with 
strong growth mindsets were able to achieve at the same level 


as high-income students with fixed mindsets.° 


In 2016, Angela Duckworth highlighted the importance of “grit” 
as apredictor of performance in Grit: The Power of Passion and 
Perseverance. Others have explored the role of broader character 
traits like perseverance, curiosity, conscientiousness, optimism, 
and self-controlin children's success. Other researchers, 
however, have questioned both the magnitude of the effect, and 


the usefulness of interventions in this area.” 


We had three objectives in reviewing the role of mindsets: to 
quantify the impact of mindsets on student performance; to 
assess which mindsets matter most; and to understand which 
types of schools and students benefit the most from certain 


mindsets. 


To quantify the impact of mindsets, we sorted the 100 most 
predictive variables (see the analytical appendix for more detail) 
emerging from the PISA surveys into anumber of specific 
categories: mindset factors, home environment (including 
socioeconomic status), school factors, teacher factors, student 
behaviors, and others. We separated mindsets into two types: 
“subject orientation” and “general mindsets.” Subject orientation 
refers to astudent’s attitudes about science as a discipline 
(science, specifically, because that was the focus of the 2015 
PISA). General mindsets refer to a student's broader sense of 


belonging, motivation, and expectations. 


To be conservative, we excluded from the analysis variables 
where we believed the direction of causality was largely from 
score-to-mindset rather than from mindset-to-score. For 
example we judged that students’ academic performance is 
more likely to influence their future educational expectations 
(whether they will complete college) than the other way around, 


and thus excluded this variable from our model. 


We then determined how influential each category was in terms 
of predicting student performance. Our conclusion: controlling 
for all other factors, student mindsets are almost twice as 
powerful (at 30 percent of total predictive power) as home 

and demographic factors? (Exhibit 11). Furthermore, general 
mindsets accounted for two-thirds ofthe effect found. The same 
pattern held true in all five regions, reinforcing the importance of 


this finding. 
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EXHIBIT 11: 

MINDSETS ECLIPSE 
EVEN HOME 
ENVIRONMENT IN 
PREDICTING 

STUDENT ACHIEVEMENT? 


Factors driving LatAm 
students OECD PISA 


science performance 
2015 


% of predictive power by catagory 
of variable 


f F 
"^ 
ir E | 
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ambitious person. ^. 
m in , 
ill help get 
have fun leaming TA 
science. Siong 
lam Interested In DAÑE h 
the universe and ie E I enoug ac 
its history. fort | can succeed. 


Air pollution will 

get worse over 

the next 20 years.” 
Nm OECD PISA 2015, McKinsey analysis 
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with a problem and is often not prepared 
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= . "Lucia wants to get top grades atschool . 
and continues working on tasks until 


everything is perfect." 
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AmO important factor. This term refers to the ability of students to => 


rrectly assess what motivation looks like, such as “working on tasks until everythingis perfect” and 


urce: "T PISA 2015, McKinsey analysis 
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mindsets are more importantthan others in 5 T 38 


expec iege To measure this, PISA.asked.test-takers to assess the motivation of , "mag 


" aUo hypothetical s students (Exhibit 13). ` 


n theresponsestothese questions, we created anindex of motivation calibration (see the — 


| T » REA appendix). What we found was that simply understanding what motivation looks like in daily 
ractice is apowerful performance indicator. Across Latin America, students who have good motivation 
f Y PRN UON score 14 ‘perce ent (or 55 PISA points) higher than poorly-calibrated students. This relationship 


| B E: A oe status, location, and type of school. In contrast, students 


ro 


tobe the best and wantingtop grades" scorejustsix percent higher than 
estan 1g top; 
l B those who do not. ads thisthecase? Our hypothesis is that students are more likely to be honest 
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EXHIBIT 14: HAVING A WELL-CALIBRATED MOTIVATION MINDSET IS 
EQUIVALENT TO LEAPFROGGING INTO A HIGHER SOCIO-ECONOMIC 
QUARTILE 


Latin America poorly performing schools, average PISA science score 


Within poor schools, 
students with low 
socioeconomic status 
and high motivation 
calibration perform 
better on PISA than 
students with high 
socioeconomic status 


calibrated.' 
Poorly Well Poorly Well 
calibrated calibrated calibrated calibrated 


1 Using PISA's economic, social, and cultural | 
status index as a proxy for socioeconomic status; | H i " 1 
looking within poorly performing schools, which Bottom socioeconomic Top BOGIGGGOLOITUG 
serve 76% of Latin American students. 

Source: OECD PISA 2015, McKinsey analysis 


quartile students quartile students 


when talking about a third person versus directly assessing their own motivation, and that calibration itself 
is actually important. Students cannot exhibit positive behaviors ifthey do not know what they look like. 
Calibrating to a norm helps to improve students’ actual study habits. 


The relationship between motivation calibration and PISA scores is twice as strong for students in poorly 
performing schools as for those in fair or good schools. (More than three-quarters of Latin American students 
are in poor schools.) In fact, for those in poorly performing schools, having a well calibrated motivation 
mindset is equivalent to vaulting into a higher socioeconomic status. Students in the lowest socioeconomic 
status quartile who are well calibrated perform better than those in the highest socioeconomic status quartile 
who are poorly calibrated (Exhibit 14). 
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EXHIBIT 15: POOR STUDENTS IN POORLY PERFORMING SCHOOLS 
ARE THE LEAST LIKELY TO HAVE GOOD MOTIVATION CALIBRATION 


96 of LatAm students with well-calibrated motivation 


E 0-49% 
E 50-59% 
, liil 60-69% 
High i 7090 
Upper middle 


ESCS Quartile 


Lower middle 


Low 


Poor Fair Good+ 


School performance level 


Unfortunately, the students in poor schools who would most benefit from high motivation calibration are 
least likely to have it— only 41 percent of low-socioeconomic status students compared with 70 percent of high 
socioeconomic status students in good schools (Exhibit 15). Girls and boys also have difference prevalence of 


motivation calibration: while 51% of Latin American girls are well-calibrated just 46% of boys are. 


These findings are consistent with those of previous PISA tests. In 2012, for example, PISA asked about 
growth versus fixed mindsets. Specifically, students answered questions about the extent to which they 


agreed that their academic results were fixed (“I do badly whether or not I study”) or could be changed 


through personal effort (“If I put in enough effort I can succeed” or “If I wanted to, I could do well”). 
Students with a strong growth mindset outperformed students with a fixed mindset by 12 percent. 
Growth mindsets were particularly predictive for students in poorly performing schools and those in 


lower socioeconomic quartiles. 
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Using the example of motivation calibration, we investigated 
how scores might improve across the region if mindsets 
could be changed. If the 51 percent of students with low 
motivation calibration could become well calibrated, and if 
the relationship between calibration and score held constant, 
outcomes would improve by 6.8 percent (28 PISA points), 


equivalent to three-quarters of a year of schooling. 


To be clear, mindsets alone cannot overcome economic and 
social barriers. This research does suggest, however, that they 
area powerful predictor of student outcomes, particularly 

for those living in the most challenging circumstances. The 
question is what, if anything, can be done to improve mindsets 


atasystem-wide level. Research is being done to answer that 
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question—albeit much of it focused on the United States—and 


there are promising indications that it may be possible for 


schools to make effective interventions. 


For example, on growth mindsets, a 2015 study of 1,500 
secondary school students in 13 different schools, rich and 
poor, from all over the United States, found that growth 
mindset and sense-of-purpose interventions delivered 
significant results. The researchers administered two 
45-minute online modules to students over the course of 
asemester. The growth-mindset modules provided direct 
instruction on the physiological growth potential ofthe brain 
given hard work; they also guided students through writing 


exercises in which they summarized what they had learned 


a Insights 


and coached a theoretical student who was losing confidence 


in his intelligence. In the sense-of-purpose module, students 
did a writingexercise on how they wished the world could be 

a better place; provided examples of why other students work 
hard; and finished with another writing exercise in which 
students explained how working hard could help them achieve 
their own goals. The results were positive: Students at risk of 
dropping out of high school, constituting a third of the sample, 
increased their grade point averages (GPA) in core academic 
courses by 0.13 to 0.18 (ona 4.0 scale), and their core course 


pass rates increased by 6.4 percent.'? 


Similarly, on motivation calibration, recent research suggests 


that meta-cognition and self-regulation strategies can 


improve student outcomes. Interventions to help students 
plan, monitor and evaluate their learning may be a promising 
way to improve student motivation and perseverance as they 


tackle challenging academic content." 


Such research is a work in progress, but these and other 
experiments indicate that harnessing the power of mindsets 
may be a promising way to support achievement—in addition, 
of course to teaching fundamental academic content. 
Academics and policy-makers in Latin America should 


be encouraged to design, implement, and evaluate further 


interventions 
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udents who receive 

a blend of inquiry-based 
and teacher-directed 
instruction have the 
best outcomes 


Teachers matter. Multiple research reports, including our 
own, have demonstrated that high-performing school systems 
require effective teachers and teaching. The challenge, then, is 
to determine what teaching practices work, and how teachers 


can deliver high-quality instruction. 


We evaluated two ty pes of science instruction to understand 
the relationship between teaching styles and student 
outcomes. The first is *teacher-directed instruction" where 
the teacher explains and demonstrates scientific ideas, 
discusses student questions, and leads class discussions. 
The second is “inquiry-based teaching," where students play 
amore active role, creating their own questions, designing 
experiments to test their hypotheses, drawing conclusions, 
and relating learning to their experiences (Exhibit 16) There 
is active debate over which approach is preferable. 
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EXHIBIT 16: OECD PISA AS 


K 
N 


) STUDEN 


o HOW OFTEN THEY 


PERIENCED THE FOLLOWING TEACHING PRACTICES 


How often does this happen in your school science class... 


Teacher-directed teaching 


* The teacher explains scientific ideas. 
A whole class discussion takes place 
with the teacher. 

* The teacher discusses our questions. 
The teacher demonstrates an idea. 


Inquiry based teaching 


e Students are given opportunities to explain their ideas. 
e Students spend time in the laboratory doing practical 
experiments. 
e Students are required to argue about science questions. 
e Students are asked to draw conclusions from an 
experiment 
The teacher explains science ideas can be applied 
Students are allowed to design their own experiments. 
There is a class debate about investigations. 
The teacher explains the relevance of concepts 
to our lives. 
e Students are asked to do an investigation to 
test ideas. 


e e eo e. 
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EXHIBIT 17: WHEN TEACHERS TAKE THE LEAD, PISA 
SCORES ARE HIGHER 


Impact of teacher-directed instruction 
LatAm average PISA science score with different amounts of teacher directed instruction' 


ll Percent of students 


> e” 
e 
"s. y t heo ght 443 
F Wwe 430 
400 | 
= 
Never Some classes Most classes All classes 


1 Trend statistically significant peaking between most and all lessons 
Source: OECD PISA 2015, McKinsey analysis 


Basedon the responses from Latin America, scores rise with increased teacher-directed instruction. 
There is an increase of 11 percent moving from a classroom where students report the use of teacher- 
directed instruction “never or hardly ever" to one where it is used “in most classes" (Exhibit 17). 


The picture for inquiry-based learning is more complex. While scores initially rise with some inquiry- 
based teaching, they then decrease with more frequent use (Exhibit 18). 


At first blush, then, inquiry-based teaching looks like a less effective choice. But when we dug into the 
data, we found a more interesting story: What matters is the interplay between the two types of teaching. 
In an ideal world, there is a place for both. Inquiry-based teaching can be effective—but only when strong 
teacher-directed instruction is in place. This suggests that teachers need to be able to clearly explain 
scientific concepts and students need to have content mastery to fully benefit from inquiry-based 
teaching. Based on the PISA results, the most effective combination appears to be teacher-directed 
instruction in most or almost all classes, with inquiry-based teaching in some of them. Students who 
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EXHIBIT 18: INQUIRY-BASED INSTRUCTION DELIVERS MIXED RESULTS 


Impact of inquiry-based instruction 
LatAm average PISA science score with different amounts of inquiry-based instructions 


Wi Percent of students 


5 


e 


^ 


434 


Gm Rf 


Never Some classes Most classes All classes 


1 Trend statistically significant except in fifth bar (most classes) which 


is not significantly different from baseline of "never or hardly ever." 
Source: OECD PISA 2015, McKinsey analysis y yA 
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EXHIBIT 19: FINDING THE SWEET SPOT: THE BEST STUDENT 
OUTCOMES COMBINE BOTH TEACHING STYLES 


Impact of teacher-direction and Inquiry-based combinations 
Expected point increase in LatAm PISA science score relative to "No classes" 
or both practice types’ 


Teacher-directed methods 


No classes Greatest expected increase (18% of students) 
Expected increase (17% of students) 
Expected decrease of 1-10 points 

(40% of students) 

Expected decrease of >10 points 

(1896 of students) 


Percent of the students 


Inquiry-based methods 


1 Statistically significant expected change in score controlling for ESCS, 
urban/rural, and private/public school 
Source: OECD PISA 2015, McKinsey analysis 


receive this blend of teaching practices outperform those who experience high levels of inquiry-based 
learning withouta strong foundation of teacher-directed instruction by 50-60 PISA points (Exhibit 19). 
To put it another way, the more teacher-directed instruction there is, the better it supports inquiry-based 
learning. 


In Latin America, most countries appear to be doing less teacher-directed instruction, and more inquiry- 
based teaching, than optimal. In fact, only 18 percent of students sit in the “sweet spot" ofteacher-directed 
instruction in most-to-all classes, supported by inquiry-based teachingin some ofthem. We estimate that 
moving the remaining 82 percent of students into the sweet spot could result in a 4.6 percent, or 19 PISA 
point, increase across the region, equivalent to about half a school year. 


This can be difficult to do. Even the best-performing school systems struggle to change teaching practices 
inthe classroom. But even just moving all students to a *low inquiry-based, high teacher-directed" style 


(the upper right corner ofthe matrix) could result in a 4.1 percent, or 17 PISA point, increase across the 
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region. This would be much easier to do, by implementing 


lesson guides and reducing the emphasis on inquiry-based 


teaching. 


These results do not take into account how good the teaching 
itselfis. There are certainly quality gaps in teacher-directed 
classrooms. The gaps are even bigger, though, in inquiry-based 
classrooms, given the need to control the necessary chaos, set 
standards and limits, monitor progress, and support students 


of different capabilities. 


Furthermore, inquiry-based and teacher-directed approaches 


are composed of specific practices, and these have discrete 


effects. In poor-performing schools in Latin America, having 
students design their own experiments had a negative effect 
on their PISA science scores, while other inquiry-based 
practices, such as applying science ideas to students’ lives, had 


noticeably positive effects ( Exhibit 20). 


This finding may seem counter-intuitive, given that there 

is strong support for inquiry-based pedagogy. We offer two 
hypotheses for why it is not translating into better student 
outcomes. First, students cannot progress to inquiry-based 
methods without a strong foundational knowledge gained 
through teacher-directed learning. Second, inquiry-based 


teaching is more challenging to deliver, and teachers who 


EXHIBIT 20: DIFFERENT TEACHING PRACTICES BRING 
DIFFERENT RESULTS 


Impact of teacher practices lil Inquiry-based 
Expected percentage increase in PISA science score E Teacher directed 
between no use and use in most classes" 


The teacher explains scientific ideas 


The teacher discusses our questions 


The teacher demonstrates an idea 


The teacher explains how a science idea can be applied 


A whole class discussion takes place with the teacher 
The teacher clearly explains relevance of science concepts to our lives 


Students are given opportunities to explain their ideas? 


Students are asked to draw conclusions from an experiment 


Students are required to argue about science questions 


Students are asked to do an investigation to test ideas 


There is a class debate about investigations 


Students are allowed to design their own experiments 


1 In a regression controlling for ESCS, public/private school and urban/rural location, normalized over regional average PISA score. 
2 For inquiry based: in most lessons; For teacher-directed: in many lessons 

3 Not statistically significant. 

Source: OECD PISA 2015, McKinsey analysis 


attempt it without sufficient training and support will struggle. This is especially true in poor-to-fair 
school systems, which account for over three quarters of Latin American schools. Our 2010 report also 
found that a more directed approach accelerated student learning for school systems in this performance 
band. 


Knowing all this is only the start, and raises a slew of questions about how to find the right balance between 
teacher-directed and inquiry-based teaching, and how to improve the quality of each. At minimum, our 
research suggests that teachers need to fully understand the content they are teaching, and be able to explain 
it, before they can jump into inquiry-based exercises. 


Well-designed classroom-based teacher coaching programs can bring results. For example, one Brazilian 
state implemented a standard teaching and supervision methodology, and provided high-quality lesson 
plans, and then supported teachers through a central and regional coaching team. The result was a 


75 percent improvement in literacy rates in four years O 
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Finding 3: 


can support student 
learning outside of 
school, its record 
inside school is mixed. 
The best results come 
when technology is 
placed in the hands 

of teachers. 


The potential of technology is obvious. It can help to 
individualize learning, assist teachers with curriculum and 
lesson plans, and equip students with the digital skills that 

will be abig part of the 21st century economy. Spending 
oninformation and communications technology (ICT) in 
education is rising; so are the hopes that ICT can help to improve 


performance. 


Several Latin American governments are investing in the 

use of ICT in the classroom. Beginning in 2011, the Brazilian 
government provided funding for 1.5 million laptops. Uruguay’s 
Plan Ceibal has provided more than 500,000 laptops to public 
primary schools. Between 2010 and 2016, the Colombian 
government delivered 2 million laptops and tablets, and has 
allocated another $25 million for tablets and laptops in schools 


through the Computadores para Educar program. 


Given all the money and attention ICT is getting, however, it 

is important to ask whether it actually improves learning. 

A 2015 OECD global report concluded that the evidence that 

it does is “mixed at best." Among countries that had invested 
heavily in ICT, the report"? concluded, there was “no appreciable 
improvements in student achievement in reading, mathematics, 
orscience." Others worry that technology in the classroom 


dehumanizes education and disempowers teachers. 


Usingthe PISA data, we explored the impact of first exposure 
to ICT, and the impact of ICT on 15-year old students at home 


and in the classroom. 
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EXHIBIT 21: EARLY EXPOSURE TO ICT IS ASSOCIATED 
WITH HIGHER SCIENCE SCORES 


How old were you when you first used a digital device? 
LatAm average PISA Science Score 


~20% improvement between first use @ Percent of students 
at under 6 versus at 13 or older! 


438 


422 
392 


Under 6 7-9 years 10-12 years 13-15 years Still haven’t 


1 Statistically significant in a regression controlling for economic, social, and cultural status; o 
public/private; and urban/rural school types. > 
Source: OECD PISA 2015, McKinsey analysis 
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Age of first ICT exposure: 


The PISA survey asked students how old they were when they 
first used a digital device or computer. Students with digital 
exposure before the age six perform about 20 percent better 


than those exposed at age 13 or later? (Exhibit 21). 


This pattern holds true regardless of socioeconomic status, 

but the effect is most pronounced for the more privileged. 
High-socioeconomic-status students who are exposed to 
digital devices at or before age six score 27 percent higher on 
PISA science than those exposed at age 18 or older. For low- 
socioeconomic status students, the difference is only 8 percent, 
and for lower- and upper-middle socioeconomic status, 13 and 
19 percent, respectively. Not only do higher-status students get 
greater lift from early use of digital devices, they are also more 
likely to have started young. The implication is that ICT may 


actually be widening the equity gap. 


It should be noted that 15-year-olds today reporting on their 
technology exposure before the age of six are referring to 
technology that is a decade old. The dynamic nature of the field 
means research like this is dated the moment it is published. 
Constant updates on the effects of technology are required to 


gainamore accurate picture. 
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EXHIBIT 22: TECHNOLOGY DIRECTED TO TEACHERS IS MOST 
EFFECTIVE AT IMPROVING LEARNING 


Impact of using specific technology use by students at school' 
LatAm percent change in PISA science score between “No” and "Yes and Use" 


E Student focussed 
Ml Teacher focussed 
E Not statistically significant 


Data-projector 


Desktop computer 


Storage 


Laptop 


Interactive whiteboard 
USB 
Tablet 


E-Book reader 


1 Statistically significant in a regression controlling for ESCS, public/private, and urban/rural school types, except for laptop effect 
Source: OECD PISA 2015, McKinsey analysis 


44 Drivers of Student Performance: Latin America Insights 


How students spend their time 
also matters. External research 
has demonstrated that going 
online for educational purposes 
and interactive game-based 
learning have positive effects, 
while participation in social 
media appears to be negative. 


ICT at home: 


Across Latin America, and after controllingfor socioeconomic 
status, schooltype, and location, students usingthe Internet 
fromtwo to four hours a day scored 46 PISA points higher than 
those with no use. More than half the benefit of Internet use at 
home—about 29 PISA points—is captured with just 31 to 60 
minutes of use per day. Beyond four hours, the positive effects 
tend to decline, and six hours ofuse or more is associated with 


negative behaviors, such as missing school. 


How students spend their time also matters. External research 
has demonstrated that going online for educational purposes 
andinteractive game-based learning have positive effects, while 
participation in social media appears to be negative not only on 
student scores, but also on student well-being." 


ICT at school: 


Regarding ICT use during school, the impact of digital 
exposure on student outcomes is more mixed. Regardless of 
the type of school or student, we found that ICT deployed to 
teachers and in support of teaching is more beneficial than 
ICT provided directly to students. For example, based upon 
the PISA principal survey, adding one data projector per 
classroom leads to a 10.4-point increase on science scores— 
equivalent to about one-third of a grade level of learning. By 
contrast, adding one student computer per classroom adds 
just 0.3 PISA points, and one teacher computer adds 2.7 
PISA points. The student survey reinforced these results. 
Again, the greatest impact came from using a data projector; 
moreover, the survey found that as currently used, access to 
some student-based technologies, such as tablets and e-book 
readers, actually seem to hurt learning (Exhibit 22). 
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Country spotlight - Mexico's IT program 


Mexico's experience demonstrates the potential benefits of combining information technology (IT) 
with a teacher- and curriculum-based approach. 


Between 2014 and 2016, the Ministry of Education began to implement @prende, a program to provide 
content across different devices to fifth- and sixth-grade students. It distributed 2 million devices, most of 


On evaluating the program, the ministry found that students in wealthy homes already owned digital 
devices and those in poorer homes could not guarantee their safety. Furthermore, teachers were unsure 
of how to integrate these tools into the classroom and were concerned that the devices themselves were 
becoming disruptive as students used them for non-academic purposes.'? These findings reinforce our 
own conclusion that technology alone does not translate into better outcomes. 


According to the PISA data, a large proportion of Mexican students have access to technology in the 
classroom. About half use Internet-connected desktop computers at school, while 20 percent use laptops, 
and 12 percent use tablets. Forty-one percent are in classrooms equipped with data-projectors. 


After controlling for socioeconomic status and school type, we found that student-centered devices, 
including tablets and e-readers, correlate negatively with academic performance in Mexico. On the 
other hand, the use of teacher-centered technology yields positive results. Exposure to data projectors, 
for example, provides a 35-point increase in the PISA science score, after controlling for socioeconomic 
status and school type. The use of desktop and Internet-connected computers, which are usually used 
in computer labs, also appear to yield slightly positive results. These patterns are consistent with our 
broader Latin American analysis. 


Based upon their own evaluations, the Mexican Ministry of Education has recently refined its approach. 
Known as (prende 2.0, the new program seeks to ensure that IT is tightly integrated with the curriculum, 
and to provide more structured computer-lab-based access to technology. It is also identifying committed 
teachers to support the rollout. 
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Given the evidence of the negligible or even negative impact 
of student-centered technology, school systems might be 
tempted to abandon their ICT efforts. Not so fast. The PISA 
survey describes the impact of education technology as 
currently implemented, not its eventual potential. First, the 
results tell us only about hardware, not software or specific 
interventions like well-executed personalized learning. 
Second, education technology is evolving rapidly and itis 
possible that specific interventions, including software and 
implementation strategies, can raise achievement at the 


system level. 


Nevertheless, Latin American school-system leaders should 
be careful not to assume that all technology is beneficial or 
even neutral to student achievement. They should work to 


ensure that ICT is fully integrated with instruction and to 


support teachers to enable them to use ICT effectively 


outcomes, up to 
seven hours per day. 
significant gains can 
also be made from 
using existing time 
better. 


The average school day in Latin America is about five 
hours, but about 15 percent of schools provide fewer than 
4.5 hours of instruction a day, and 6 percent provide more 
than seven hours. There are also significant variations from 


country to country (Exhibit 23). 


Many Latin America countries, then, are attempting to 
lengthen the school day. In the Dominican Republic, 73 
percent of primary schools and 65 percent of secondary 
schools still operate on a shift basis, with two sets of students 
sharingthe same facilities daily. In Brazil, several states 

are attempting to eliminate evening instruction, in which 
secondary school students use primary school facilities in the 
evening. In 2015, the Colombian government began Jornada 


Unica, an initiative to extend the school day to seven hours.'^ 
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EXHIBIT 23: THE AMOUNT OF TIME SPENT 
IN SCHOOL VARIES SIGNIFICANTLY 


Average reported school day by country; Hours per day! 


Costa Rica 


Peru 


Mexico 


Colombia 


Brazil 


Dominican Republic 


1 Instructional hours per day at the school level, reported by students; data unavailable for Argentina and Trini- 
dad & Tobago;Includes Chile, Costa Rica, Peru, Mexico, Colombia, Brazil, Dominican Republic, and Uruguay 
Source: OECD PISA 2015, McKinsey analysis 


A 


Average for Latin America 


pF 
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EXHIBIT 24: EACH ADDITIONAL HALF HOUR OF INSTRUCTION UP 
TO 6.5 TO 7 HOURS PER DAY IMPROVES STUDENT OUTCOMES 


PISA score benefit of longer school days! 
LatAm PISA science score 2015, by daily hours of instruction 


3-3.5 3.5-4 4-4.5 4.5-5 5-5.5 5.5-6 6-6.5 6.5-7 7-7.5 


1 All increases statistically significant, includes Chile, Costa Rica, Peru, Mexico, Colombia, Brazil, Dominican Republic, and Uruguay 
Source: OECD PISA 2015, McKinsey analysis 


The striking variation in the length ofthe school day in Latin America can help us understand the 
implications of increasing the hours of instruction. We looked to the PISA data set to understand 
the academic impact of each half hour of additional instruction. 


It makes intuitive sense that spending more time in school should improve academic outcomes, 

and the PISA results bear that out. Across Latin America, PISA science scores increase 3.7 percent 
(or 14 PISA points) for every 30 minutes of additional daily classroom instruction, up to seven hours 
per day. Additional time beyond 6.5 to 7 hours per day does not lead to higher scores, suggesting 
diminishing returns as the limits of student stamina are reached. This relationship holds, even after 
controlling for economic, social, and cultural status, and type of school (Exhibit 24). 


Drivers of Student Performance: Latin America Insights 


49 


122 
120 
118 
16 
14 
12 
110 
8 

6 

4 

2 


| IIl 


Score per hour 


Score per hour W Learning time at school W Latin American countries 


MOST LATIN AMERICAN COUNTRIES DO NOT USE 


THEIR CLASSROOM TIME PRODUCTIVELY 


In-school learning time and science points per hour of learning time 


35. Hours per week 
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Source: OECD PISA 2015, McKinsey analysis 
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Ifall students received similar outcomes to those currently 
receiving 6.5 to 7 hours of instruction per day, average PISA 
science scores across the region would increase by 35 PISA 
points. But there are no guarantees that the relationship with 
score would hold, and increasing hours of instruction would 
be costly. In addition to having to pay for additional teaching 
time, extending the school day could require new buildings 
and other infrastructure. One alternative to longer school 
days is to impose a longer school year. Research in Mexico 
found that adding more days of instruction slightly improved 
student performance, but exhibited diminishing marginal 
returns with lower improvements in poorer schools." This 
suggests that, at present levels of quality, increasing the 


number of days spent in the classroom is not enough. 


How time is used is therefore critical. The OECD recommends 
that 85 percent of classroom time be used for learning; no 
Latin American country reaches this benchmark. Colombia 


wasthe closest (65 percent), followed by Brazil (64 percent) 
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and Honduras (64 percent).'* Itis difficult to measure the 
quality oflearningtime, but one useful metric is PISA points 
per hour of instruction. By this admittedly limited standard, 
Latin American school systems fall short, comprising five 
ofthe ten least productive countries in the PISA data set 
(Exhibit 25). 


Educators, then, have to make a trade-off, at least in the 

short- to medium-term, between investing to increase 

the number oftotal school day hours versus getting more 

and better instruction out of existing time. By raising the 
share of classroom time used for learning, and improving 
instruction quality through teacher coaching and professional 
development, Latin America could significantly improve 


student learning for every existing hour in school ri 
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Country spotlight: A cost-benefit analysis of 
increased hours of instruction in Brazil 


In Brazil, about half of schools provide fewer than five hours of instruction, and many of them share their 
buildings—for example, with primary schools in the morning and secondary schools in the afternoon. 
Several regions, including Pernambuco and Sáo Paulo", are trying to extend the school day, but the jury is 


still outonthe relative costs and benefits of these interventions. 


Looking at the PISA Brazil data, it appears there could be real benefits to a longer school day; every 
additional half hour of instruction between 3 and 6.5 hours provides a 2.9 percent increase in the PISA 
science score. This pattern holds controlling for socioeconomic status and type of school, with the highest 


scores achieved by students in schools with between 6.5 and 7 hours of instruction. 


The costs, however, are high. Using existing infrastructure, adding additional hours of instruction to 
bring every part-time student up to 5 hours per day would cost $530 million to $610 million”, or about 0.6 
percent of total public education spending. Assuming that the relationship between hours of instruction 


and score holds, that would lead to a 2.8 percent net improvement in national PISA scores. 


If additional infrastructure is required, the costs increase significantly. Changing all existing shift schools 
into full-time programs with 6.5 hours of instruction would cost between $3.1 billion to $3.6 billion, or 
3.8 percent of total public education spending.” This could improve national PISA scores by up to 4.1 
percent.?? [n either case, the efforts could fall short if the additional hours are not used well, in terms of 


teachers and instructional content. 
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Finding 5 


education had a 
positive academic 
impact on today's 

15 year olds, however 
low income students 
received less benefit 
than high income 
ones 


More than half of the synaptic connections that allow 
people to think, see, hear, and speak are formed before age 
three.” Although brain plasticity persists into adulthood, the 
brain is most receptive to interventions in early childhood. 
That is the promise of quality early-childhood education 
(ECE), and indeed such programs have been shown to improve 
academic and social outcomes, especially for disadvantaged 
children.?* Although there are some concerns about fade- 

out in later years, good ECE programs can help to narrow 

the achievement gap by helping disadvantaged children gain 


cognitive, social, and other skills before starting kindergarten. 


Many Latin American governments recognize the potential 
of ECE. For example, Brazil has made this a priority and is on 
trackto meet its goal of universal ECE for four-year-olds by 
2024. The challenge is how to capture the benefits of ECE ina 


cost-effective manner, and how to balance access with quality. 


The PISA survey asked students how old they were when 

they started formal education. Parents in Chile, Dominican 
Republic, and Mexico also answered detailed questions about 
their children's early education. On the basis of these answers, 
our findings highlight the trade-off between increasing access 


and ensuring quality. 
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EXHIBIT 26: ECE DOES NOT APPEAR TO WORK AS WELL FOR LOWER 
SOCIOECONOMIC STATUS STUDENTS IN LATIN AMERICA 


Impact of ECE on score Students with no ECE 


Increase in PISA Science Score Percent of students 


More students 
without early 


High socioeconomic status +11% 7% childhood education 
Upper middle socioeconomic status +7% 11% 
Lower middle socioeconomic status +5% 13% 
Low socioeconomic status +4% Higher 16% 
impact for 
higher ESCS 


1 Statistically significant in regression with standard controls, when compared against scores of students with no early childhood exposure 
Source: OECD PISA 2015, McKinsey analysis 


Across Latin America, 78 percent of students told PISA they had received some form of ECE by age five. 
Those with some ECE scored eight percent (or 30 PISA points) higher on the PISA science assessment 
than those with none. Controlling for student socioeconomic status, and school location and type 
students still get a benefit of 8 PISA points. While students with ECE perform better overall, there are 
substantial differences between students from different socioeconomic backgrounds. Specifically, early- 
childhood programs do not appear to be narrowingthe achievement gap between lower- and higher- 
socioeconomic status students. Poorer students are less likely to get ECE in the first place, and those who 
do see a smaller impact on their PISA scores (Exhibit 26). 


Even more troublingis the fact that low-socioeconomic-status children who start ECE at three years or 
younger actually do worse than those who start at age four or five. In contrast, high-socioeconomic-status 
children do best when they start ECE at age two or three (Exhibit 27). 


This raises serious questions about the quality of ECE available to lower-socioeconomic-status children. 


A recent Inter-American Development Bank (IDB) report” highlighted the importance of quality care 
and found that programs for younger children sometimes led to worse outcomes than no ECE at all. 
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EXHIBIT 27: HIGH SOCIOECONOMIC STATUS CHILDREN BENEFIT MOST 
FROM STARTING ECE AT AROUND AGE TWO; LOW SOCIOECONOMIC STATUS 
CHILDREN DO NOT SEEM TO BENEFIT FROM STARTING EARLY 


(B Percent of Latin American students 


Average PISA score by age started early education 


High ESCS: "T 


Low ESCS: 
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Conclusion 


Our research has mapped some areas previously 

blank, and also identified new territories worthy of further 
exploration. For each of the five findings, there is aclear 

need for additional research. Within mindsets, the priority 

is to determine what system-level interventions can make 

a difference in shifting student mindsets, and what effect 
these interventions have on student outcomes. For teaching 
practices, more research is needed into how to effectively 
combine teacher-directed and inquiry-based learning. In ICT, 
we need more rigorous longitudinal studies that consider not 
only what hardware works, but also what software and system 
supports lead to successful outcomes. Across the board, more 
research is needed on how to strike the right balance between 
increasing access and improving quality. This is particularly 
relevant in terms of increasing hours of instruction in systems 
with shared infrastructure and in rolling out government- 
provided ECE. In an important sense, then, this report—like 


our previous two—is part of a longer journey. 


With its emphasis on data and analysis, this research aims 

to help Latin American school systems move from poor to 
fair to goodand beyond. Even asurvey as large and rigorous 
as the PISA data set provides only some of the answers. But 
we believe that the five findings outlined here, combined 
with the conclusions of our 2010 report on the world’s most 
improved school systems, provide useful insights to guide 
Latin American policymakers as they make their way to their 
ultimate destination—improving the education and thus the 


lives of the region’s students M 
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analvtical 
appendix 


To analyze the PISA dataset, 
we used a variety of 

modern machine learning 
and traditional statistical 
techniques. 


First, we used SparkBeyond, an automated feature- 
discovery engine that uses large-scale combinatorial 
testing of millions of transformations on raw data to 
identify relevant drivers of outcomes—in our case, PISA 
student scores. SparkBeyond can create features from 
numeric, time series, text, and other inputs, and works 
best with complex data sets with thousands of variables 
and millions of data points. For the 2015 OECD PISA data, 
this entailed testing more than 1,000 survey variables 
derived from student, teacher, parent, and principal 
surveys for the approximately 540,000 students who 
took the PISA examination. This identified variables and 
groups of variables that were most predictive of student 
performance. 


We excluded from our SparkBeyond and subsequent 
analysis highly predictive variables where the direction 

of causality was strongly in question, including grade 
repetition, student self-efficacy, environmental awareness, 
expected educational attainment, and epistemological 
beliefs. 


We then carried out traditional descriptive and predictive 
statistical analyses on the identified features that were 
most important in determining performance both within 
2015 dataset and across the PISA surveys since 2000. 


For every analysis, we tested whether findings held in a 
regression controlling for economic, social, and cultural 
status (ESCS), type of school (SC013Q01: is your school 

a public or private school school?) and location of school 
(SC001Q01: which of the following definitions best 
describes the community in which your school is located?). 


Where the regression results were consistent with the 
descriptive analysis, we have used the descriptive analysis 
in the report. Where the regression tells a different 

story from the description, we have reported regression 


coefficients to preserve the rigor of our findings. 


School Performance Levels 


V 
o 
o 
= 


A A A 


< Fair >< Good >< Great ><— Excellent —> 
440-480 480-520 520-560 >560 points 


<440 points 


E the 20 1 5 OECD We also tested our insights by school and student 


segment, creating two more screens—specifically, school 


PISA data, this entailed performance level and student socioeconomic status. 
testing More than | OOO School performance: 
S U (VEY Variables de rived we used the numerical cut-offs from our 2010 report 


to define poor, fair, good, great, and excellent school 


tro Li SUE e nt , ¡556 h = [E systems. Each cana represents jT. one 
parent cric 9 ri N Cl pal school-year equivalent, or 40 PISA points. 


e Excellent: 2560 points 


Surveys Te the e Great: 520-560 points 


approximately 540,000 — raraso-aeo points 
students Who took * Poor: «440 points 


the P| A exam | nati Of) Then we applied these cut-offs to individual schools as 


well as to school systems. We did this because there may 
be pockets of poorly performing schools in otherwise 


good systems. In these schools, the interventions 
applicable to poor systems may apply, even if they are 

in a country that on the whole performs at a “good” 

level. Based on this analysis, we could determine the 
percentage of students in differently performing schools 
for each region and country 


Student socioeconomic status: 

We use the term *student-socioeconomic-status quartile" 
throughout the report. This refers to PISA's ESCS indicator 
that integrates a number of measures related to students’ 
backgrounds, including their parents' occupations, 
education levels, and possessions. We created ESCS 
quartiles by region based upon student weights. 


Target variables and plausible values 


We used the 2015 PISA science score as the target 
variable because the 2015 test focused on science both 
for the assessment and survey questions (in 2012, PISA 
focused on math, and in 2009, on reading). To calculate the 
PISA science score at the student level, we averaged the 
results of all the plausible values for science (PV1 to PV10 
for science). 


To roll up scores at the regional level, we used student 
weights to represent each country based on its student 
population. For example, the Latin American numbers all 
refer to weighted average student scores across Latin 
America; the same is true for all other regions. 


For consistency with OECD publications, we used a 
slightly different methodology in the overview of historical 
regional performance. This approach creates a country- 
level average, first using student weights (such as 
*average score for Brazil"), but then takes the straight 
average of the scores of countries in a particular region 


or a group (such as "all OECD countries"). 


Description of specific variables 


In addition to using existing OECD PISA variables and 
indices, we created our own indices for some analyses. 


Motivation calibration: 

Motivation calibration is a measure of a student's ability to 
recognize motivation in others, or the extent to which the 
student's definition of motivation agrees with the standard 
definition. Specifically, we took the PISA question ST121, 
which presented three student archetypes and asked the 
respondent to what extent they agree that each archetype 
is motivated on a four-point scale, ranging from “strongly 
disagree" to *strongly agree." 


Based on our assessment of the motivation level of each 
archetype, we assigned a weight of -2 to the first student 
(NAME 1—highly unmotivated), +1 to the second student 
(NAME 2—somewhat motivated), and +2 to the third 
student (NAME 3—highly motivated). 


For example, a student who strongly disagreed that 

«NAME 1> is motivated, agreed that «NAME 2» is 

motivated, and strongly agreed that «NAME 3» is 

motivated would accumulate the following score: 

e 1*-2=-2: one point for strongly disagree with a 
weight of -2 for <NAME 1> 

e 3*1 = 3: three points for agree with a weight of 1 
for «NAME 2> 

e 4* 2 = 8: four points for strongly agree with a weight 
of 2 for c. NAME 3» 

e Total score: -2+348=9 


Please read the descriptions about the following three students. Based on 
the information provided here, how much would you disagree or agree with the 


ST121 


statement that this student is motivated? (Please select one response in each row.) 


Strongly Disagree Strongly Agree 
disagree Agree 


ST121Q01NA <NAME 1> gives up easily when confronted with 
a problem and is often not prepared for his 


classes. <Name 1> is motivated. 


<NAME 2> mostly remains ir 
she starts a 
expected from her. <Name 2> is motivated 


ST121Q02NA 


«NAME 3» wi o get top grades at school 
and continu vorsing on tasks until everything 
is perfect, «Name 3> is motivated 


ST121Q03NA 


Thinking about your school: to what extent do you agree with 
STO34 the following statements? (Please select one response in each row.) 


Strongly Disagree Strongly Agree 
disagree Agree 


STO34Q01TA _ | feel like an outsider (or left out of things) at school. 1 


STOS4Q02TA | make friends easily at school. 1 3 4 


STOS4QOSTA _ | feel like | belong at school 


STO84Q04TA  |feel awkward and out of place in my school 1 3 4 


STO84Q05TA Other students seem to like me 1 


STO34Q06TA  Ifeel lonely at school 1 


To what extent do you disagree or agree with the following statements 
ST1 1 9 about yourself? (Please select one response in each row.) 


Strongly Disagree Strongly Agree 


Agree 
ST119Q01NA | want top grades in most or all of my courses. 2 4 
ST119Q02NA _ | want to be able to select from among the best 
opportunities available when | graduate. 
ST119Q03NA | want to be the best, whatever | do 2 y 4 
ST119QO04NA | see myself as an ambitious person. 1 > 3 4 
ST119Q05NA _ | want to be one of the best students in my class. 


We defined a cutoff of 8 points in the aggregated 
score, which ensures that only the following 
students are classified as having a strong 
motivation calibration: 


e Students who strongly agree that <NAME 
3> is motivated, and whose agreement on 
«NAME 1>’s motivation does not exceed their 
agreement on «NAME 2>’s motivation 


e -OR— Students who agree that «NAME 3>’s is 
motivated; agree that «NAME 2» is motivated, 
and strongly disagree that «NAME 1> is 
motivated 


e -OR— Students who agree that «NAME 3» is 
motivated; strongly agree that «NAME 2» is 
motivated, and disagree or strongly disagree 
that «Name 1» is motivated 


Sense of belonging: 

We grouped the index BELONG (based 
on ST034) as follows: 

* Low belonging: BELONG « 0 

* High belonging: BELONG >=0 


Motivation: 

We grouped the index MOTIVAT (based on 
ST119) as follows: 

* Low belonging: MOTIVAT « 0 

* High belonging: MOTIVAT >=0 


Test anxiety: 

We grouped the index ANXTEST (based on 
ST118) as follows: 

* Low belonging: ANXTEST « 0 

* High belonging: ANXTEST >=0 


Instrumental motivation: 

We grouped the index INSTSCIE (based on 
ST113) as follows: 

* Low instrumental motivation: INSTSCIE « 0 
* High instrumental motivation: INSTSCIE >=0 


Growth vs. fixed mindset: 

To assess the impact of a growth versus fixed 
mindset, we used selected 2012 PISA survey 
question ST43 and ST91 from the student 
survey. 


We created an index by adding the response 
values for each of the four sub-questions related 
to growth versus fixed mindsets, after reversing 
the sequence of response values for the last 
question to account for the negative framing of 
the prompt. 


The resulting index takes values from 4 to 16, 
with lower scores representing a growth mindset 
and higher scores representing a fixed mindset. 
Looking at the distribution of students globally, 
we devised the following definitions. 


e Strong growth mindset: students with a 
score of 4 or 5 reflect a growth mindset on 
atleast three of the sub-questions, and 
are directionally aligned on the remaining 
question. These represent 23 percent of the 
global population. 


* Neutral or weak growth mindset: students 
with a score of 6 to 9 reflect a neutral or weak 
growth mindset and represent 69 percent of 


the global population. 


"Y Y lu i 


To what extent do you disagree or agree with the following statements 
about yourself? (Please select one response in each row.) 


ST118 


Strongly Disagree Strongly Agree 
disagree Agree 
ST118QO01NA loften worry that it will be difficult for me taking a test. 1 ; 3 
ST118Q02NA | worry that | will get poor «grades» at school. 1 
ST118QO03NA Even ifl am well prepared for a test | feel very anxious. 1 2 
ST118Q04NA | get very tense when | study for a test 1 2 
ST118Q05NA _ | get nervous when | don't know how to solve a task 
at school. 1 4 
How much do you agree with the statements below? 
(Please select one response in each row.) 
Strongly Disagree Strongly Agree 
disagree Agree 
ST118Q01TA Making an effort in my «school science» subject(s) is 1 
worth it because this will help me in the work | want 
to do later on. 
ST118Q02TA What | learn in my «school science» subject(s) is 1 
important for me because | need this for what | want 
to do later on. 
ST113Q03TA Studying my «school science» subject(s) is 1 
worthwhile for me because what | learn will improve 
my career prospects 
ST118Q04TA Many things | learn in my «school science» subject(s) n A " 
will help me to get a job. 
Thinking about your math lessons: to what extent do you agree with 
ST043 the following statements? (Please select one response in each row.) 
Strongly Disagree Strongly Agree 


disagree Agree 


(a) If | put in enough effort | can succeed in mathematics 


(b) Whether or not | do well in maths is up to me ` 2 3 


(c) f| wanted to, | could do well in mathematics ‘ 2 3 


(d) | do badly in mathematics whether or not | study for my exams ] 3 


The goal of the following set of questions is to gather information 
SC004 about the student-computer ratio for students in the <national modal 

grade for 15-year-olds> at your school. 

(Please enter a number for each response. Enter "O" [zero] if there are none.) 


Number 


SCOO4QO1TA At your school, what is the total number of students in the «national modal * Fixed mindset: students with a score of 10 to 


grade for 15-year-olds»? 16 have an average response of 2.5 or more on 


SC004Q02TA Approximately, how many computers are available for these students for the four questions, meaning that they tend to 


educational purposes? 
SCOO4QOSTA Approximately, how many of these computers are connected to the Intemet/ be misaligned with the principles ofa growth 


World Wide Web? mindset. They represent 8 percent of the 
SCO04Q04NA Approximately, how many of these computers are portable (e.g. laptop, tablet)? lobalipooulation 
SCO04QO05NA Approximately how many interactive whiteboards are available in the school g pop p 
altogether? 
SCO04Q06NA Approximately how many data projectors are available in the school altogether? We compared students with a fixed mindset to 
SCO04Q07NA Approximately how many computers with internet connection are available for 


teh HT students with a strong growth mindset in our 
eachers in your school? 


analysis. In addition, we found that incremental 
gains were seen at each stage from fixed to 


neutral and from weak growth to strong growth. 


How old were you when you started <ISCED 0>? 
ST1 25 (Please choose from the drop-down menu to answer the question.) 


Teaching practices: 
To assess teaching practices, the PISA survey 


asked a series of questions about teacher- 


Years | Please choose Y | 
pion directed instruction (ST103) and inquiry-based 
Option B instruction (ST098). This question does not 
Option C allow us to assess the intensity of the teaching 
| Option practices in a given class, but only the frequency 


with which they occur. 

Drop-down menu, offering answers “1 year or younger’, 2 years, 3 years, 4 years, 5 years, "6 years 

or older", "| did not attend «ISCED O>", “| do not remember". Students responded ona frequency scale that 
was slightly different for each set of questions: 


Teacher-directed learning (ST103) 

1 = Never or almost never 

2 = Some lessons 

3 = Many lessons 

4 = Every lesson or almost every lesson 


Inquiry-based learning (STO98) 
1 = In all lessons 

2 = In most lessons 

3 = In some lessons 


4 = Never or hardly ever 


We consolidated each student's responses into averages 
ona scale from 1 to 4—one average for teacher-directed 
instruction and another for inquiry-based instruction (with 
the numbers reversed to be comparable). These averages 
form the basis for our analysis of teaching practices. 


The OECD also created a numerical index of teacher- 
directed (TDTEACH) and inquiry-based learning (IBTEACH), 
which is calibrated such that the OECD average is 0 and 

the standard deviation is 1. When we ran regressions on 

the TDTEACH and IBTEACH variables, our results were 
consistent with theirs. However, we chose to present the 
data using our own indices because we believed these gave 
a clearer picture what was happening in the classroom. 


ICT at school: 

to create a like-for-like comparison of the impact of ICT 
hardware, we used the survey questions asked of school 
principals from SC004 and normalized the results by 
classroom size and student-to-teacher ratio. This allowed 
us to evaluate the effect adding one projector, student 
computer, or teacher computer to an average class size of 
36 students. 


Early childhood: 

To understand the impact of early-childhood education 
(ECE) we used the student survey question ST125. We 
excluded from the analysis students who could not 
remember when they started ECE. With the remaining 
students, we counted them as having attended ECE if they 
started at five years or younger. Students who started at 
six years or older or who responded “no early-childhood 
education" we counted as not having attended ECE. Note 
we did not use the simpler question ST124 (“Did you attend 
early-childhood education,” as only 15 percent of students 
globally answered this question (versus 82 percent who 
answered ST125). We also cross-checked results against 
similar questions in the parent survey for the subset of 
countries that took the parent survey; the results were 


consistent B 


1 These five stages inform McKinsey's Universal Scale of education system performance, which takes available assessments 
like PISA, TIMSS, TERCE, or local tests. We normalized the data, creating new units that are equivalent to 2000 PISA 
Scores, and then broke down the results into five categories: poor, fair, good, great and excellent (see the analytical 
appendix for more, and also http://www.mckinsey.com/industries/social-sector/our-insights/how-the-worlds-most- 
improved-school-systems-keep-getting-better). 


2 The 2015 computer-based assessment was designed as a two-hour test comprising four 30-minute clusters. Students 
took two science clusters, plus two others across reading, math, and collaborative problem-solving. 


3 Argentina, Kazakhstan, and Malaysia were excluded from PISA 2015 report, but are included in our analyses. The PISA 
2015 sample for Malaysia did not meet PISA response-rate standards; the PISA 2015 sample from Argentina did not cover 
the full target population; and the results from Kazakhstan are based only on multiple-choice items. Because our report 
analyzes achievement drivers at the student level based on examining individual items, and reports are generated at a 
regional basis rather than comparing the performance of individual countries, we have included these countries in our 
analysis. Albania was excluded from our analysis because, due to the ways in which the data were captured, it was not 
possible to match the data in the test with the data from the student questionnaire. As our report is entirely based on th 
drivers from the student questionnaire, we could not include Albania in our analysis. 


We used the full Argentina results, despite some concerns about data reliability, because they were more reflective of full- 
country performance than the Buenos Aries (CABA) results. Official OECD reports include only the CABA results. 


5 The OECD measures equity in terms of the percent of score variation that depends on student socioeconomic status, 
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